---
title: "What In The Hell Did I Write In Here Part2"
date: 2021-02-20T06:31:05Z
description: "Continuation of the RSS post"
keywords: [rss, linux, python]
tags: [linux, python]
categories: python
---

I just realized I didn't finish my "What In The Hell Did I Write In Here" post.
It was my intention to document and comment all the RSS app's modules in a blog post.
So not to look at  it as if it was something you found in a museum, with a tag saying:
"Unknown Origin. Age: Undetermined. Usage: Some sort of chamber pot, or possibly a drinking vessel.
Maybe both."
I meant to do it, but then I changed my approach to the blog, and this was put in the,
"to do when you're not thinking about anything.", pile.
As I currently am, very actively, not thinking, and totally bereft of thought,
makes of this, the correct moment to resume this endeavor.

<h3>append_rss.py</h3>

This module is necessary because, in insert_db.py, the line
`fp = feedparser.parse(url)` ,
wouldn't accept a function object,
which is what happens if you imported the outcome of this module directly.
So, it was necessary to assure that the URL would be formatted as a string.
Hence the passage to a text file.
This has the added advantage of creating a record of all feeds in use.
``` python
    import click

    def append_rss():
        new_rss = input(click.style(' What is the url you want to add? ', fg='magenta', bold=True))
        envio = open('url_list.txt', 'a')
        envio.write(new_rss)
        envio.close()

    if __name__ == "__main__":
        append_rss()
```

<h3>delete_rss.py</h3>

Since a text file was created in the last module with a list of the URLs of the feeds in use
by the app, it stands to reason that, if wanted, we would edit this file to delete a feed.

- We first show the user the available feeds in file `lista`:
     ```python
     def read_rss():
        with open('url_list.txt') as f:
             lista = f.read()
             print(lista)
    ```
- Then we create another function to house the deletion command.
```python
     def delete_rss():
```
- You need to open the file and read its contents in memory,
     then open the file again write the line to it but without
     the line you wish to omit.
```python
         delete_rss = input(click.style(' What is the url you want to delete? ', fg='magenta', bold=True))
         with open('url_list.txt', 'r') as f:
             lines = f.readlines()
         with open('url_list.txt', 'w') as f:
             for line in lines:
                 if line.strip('\n') != delete_rss:
                     f.write(line)


    if __name__ == "__main__":
        delete_rss()
```

### insert_db.py

In order to keep our database clean and up to date we'll first delete old
entries in the database, and create new ones.
- From [dateutil] [date], a library for date manipulations, we'll import parse
  we also need SQLite for db access and [feedparser] [feed] for the RSS.
```python
    from dateutil.parser import parse
    import sqlite3
   import feedparser
```
- Open a SQLite connector and erase all entries in the table,
```python
        def delete_old():
            with sqlite3.connect('rss.db') as db:
                cur = db.cursor()
                inserir = 'DELETE FROM rss;'
                cur.execute(inserir,),
                db.commit()
```
- Now we define a function containing the URL of the rss',
  after we define what elements of the feed we are going to
  use.
```python
        def get_rss():
```
- We read the file into a list of lines,
```python
        with open('url_list.txt') as f:
            urls = f.read().splitlines()
```
- define 0 as the start of our index,
```python
        index = 0
```
- connect to the db,
```python
        with open('url_list.txt') as f:
            urls = f.read().splitlines()
```
- create feedparser object for each publication in the file list,
```python
        for url in urls:
            fp = feedparser.parse(url)
            nome = fp['feed']['title']
            print(nome)
```
- for each unique number that defines each content piece,
  we take out the ones that have a item named 'title' and
  'link' and turn them to strings.
  As not all publications had the 'published' item, it was
  necessary to place its extraction inside a
  'try/except' element, or the module would stop with key
  and attribute errors.
  Also, as it is a time element, the date format imported
  by the feed is not valid in SQLite3.
  So it was necessary to change it to something more
  acceptable.
```python
        for index in range(len(fp.entries)):
            try:
                titulo = fp.entries[index].title
                titulo = str(titulo)
                linque = fp.entries[index].link
                linque = str(linque)
                publi = fp.entries[index].published
                publi = str(publi)
                pub = parse(publi)
                tempo = pub.strftime('%y/%m/%d')
        except KeyError:
```
- As the Reddit rss' don't have a published element,
  but have a updated one,
  there was a need to create another if-exception block,
  so he could look on both.
```python
                fp.entries[index].updated
                publi = fp.entries[index].updated
                publi = str(publi)
                pub = parse(publi)
                tempo = pub.strftime('%y/%m/%d')
                except KeyError:
                pass
            except AttributeError:
                pass
            try:
                fp.entries[index].pubDate
                publi = fp.entries[index].pubDate
                publi = str(publi)
                pub = parse(publi)
                tempo = pub.strftime('%y/%m/%d')
            except KeyError:
                pass
            except AttributeError:
                pass
        except AttributeError:
            try:
                fp.entries[index].updated
                publi = fp.entries[index].updated
                publi = str(publi)
                pub = parse(publi)
                tempo = pub.strftime('%y/%m/%d')
            except KeyError:
                pass
            except AttributeError:
                pass
        try:
```
- We read the file into a list,
```python
        fp.entries[index].pubDate
        publi = fp.entries[index].pubDate
        publi = str(publi)
        pub = parse(publi)
        tempo = pub.strftime('%y/%m/%d')
    except KeyError:
        pass
    except AttributeError:
        pass
```
- finally we insert the rss elements to the db and
  close the connection.
```python
        inserir = 'INSERT INTO rss (name, title, link, date) VALUES (?, ?, ? , ?)'
        cur.execute(inserir, (nome, titulo, linque, tempo)),
        db.commit()
```
- The complete code.
```python
        from dateutil.parser import parse
        import sqlite3
        import feedparser


        def delete_old():
            """Opens a sqlite3 connection and erases all entries in the table 'rss'"""
            with sqlite3.connect('rss.db') as db:
                cur = db.cursor()
                inserir = 'DELETE FROM rss;'
                cur.execute(inserir,),
               db.commit()


        if __name__ == "__main__":
            delete_old()


        def get_rss():
            """Contains the code that defines the url of the rss, after we define what
             elements of the feed want."""

            with open('url_list.txt') as f:
                urls = f.read().splitlines()

            index = 0

            with sqlite3.connect('rss.db') as db:
                cur = db.cursor()
                for url in urls:
                    fp = feedparser.parse(url)
                    nome = fp['feed']['title']
                    print(nome)
                    for index in range(len(fp.entries)):
                        try:
                            titulo = fp.entries[index].title
                            titulo = str(titulo)
                            linque = fp.entries[index].link
                            linque = str(linque)
                            publi = fp.entries[index].published
                            publi = str(publi)
                            pub = parse(publi)
                            tempo = pub.strftime('%y/%m/%d')
                        except KeyError:
                            try:
                                fp.entries[index].updated
                                publi = fp.entries[index].updated
                                publi = str(publi)
                                pub = parse(publi)
                                tempo = pub.strftime('%y/%m/%d')
                            except KeyError:
                                pass
                            except AttributeError:
                               pass
                            try:
                                fp.entries[index].pubDate
                                publi = fp.entries[index].pubDate
                                publi = str(publi)
                                pub = parse(publi)
                                tempo = pub.strftime('%y/%m/%d')
                            except KeyError:
                                pass
                            except AttributeError:
                                pass
                        except AttributeError:
                            try:
                                fp.entries[index].updated
                                publi = fp.entries[index].updated
                                publi = str(publi)
                                pub = parse(publi)
                                tempo = pub.strftime('%y/%m/%d')
                            except KeyError:
                                pass
                            except AttributeError:
                                pass
                            try:
                                fp.entries[index].pubDate
                                publi = fp.entries[index].pubDate
                                publi = str(publi)
                                pub = parse(publi)
                                tempo = pub.strftime('%y/%m/%d')
                            except KeyError:
                                pass
                            except AttributeError:
                                pass

                        inserir = 'INSERT INTO rss (name, title, link, date) VALUES (?, ?, ? , ?)'
                        cur.execute(inserir, (nome, titulo, linque, tempo)),
                        db.commit()


      if __name__ == "__main__":
          get_rss()
```

<h3>show_rss.py</h3>

This module shows the content of the db. Turned it to a module,
so I can separate changing the db and seeing it.

- In essence, you just open the db and embellish the output.
```python
         def show_rss():
         with sqlite3.connect('rss.db') as db:
            cur = db.cursor()
            cur.execute('SELECT * FROM rss ORDER BY strftime(date) ASC')  # 1
            rows = cur.fetchall()
            for row in rows:
                print(click.style('Ж - ', fg='bright_blue', bold=True), click.style(str(row[1]), fg='bright_blue', bold=True))
                print(click.style('Щ - ', fg='bright_cyan', bold=True), click.style(str(row[2]), fg='bright_cyan', bold=True))
                print(click.style('Ÿ - ', fg='bright_blue', bold=True), click.style(str(row[3]), fg='bright_blue', bold=True))
                print(click.style('Ʞ -', fg='bright_cyan', bold=True), click.style(str(row[4]), fg='bright_cyan', bold=True))
                print('\n')


       if __name__ == "__main___":
          show_rss()
```

### sources_search.py

- Internal function that calls the db by publication. It's
  called from the main module.
```python
        import click
        import sqlite3

        source = ''


       def sources(source):
       """Getting its query from main, the function calls the db, and asks for rss_fts
          table, which has free text search enabled."""
       with sqlite3.connect('rss.db') as db:
            cur = db.cursor()
            expression = 'SELECT * FROM rss_fts WHERE rss_fts MATCH ? ORDER BY date DESC'
            cur.execute(expression, (source,))
            record = cur.fetchall()
            for row in record:
                print(click.style(' 𝝵 ', fg='bright_magenta', bold=True),
                      (click.style(str(row[1]), fg='bright_magenta', bold=True)))
                print(click.style(' 𝞇 ', fg='bright_magenta', bold=True),
                      (click.style(str(row[2]), fg='bright_blue', bold=True)))
               print(click.style(' 𝝮 ', fg='bright_magenta', bold=True),
                      (click.style(str(row[3]), fg='bright_white', bold=True)))
               print('\n')


        if __name__ == "__main__":
             sources(source)
```

And that is all the code in my RSS app.
I'm always thinking that I should add
some bells and whistles, make it more
rounded. But the truth is, for my use,
it doesn't need anything else.


[date]: https://dateutil.readthedocs.io/en/stable/parser.html
[feed]: https://pythonhosted.org/feedparser/
